Papers with monosemanticity probing
Encourage or Inhibit Monosemanticity? Revisit Monosemanticity from a Feature Decorrelation Perspective (2024.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies focus on monosemanticity on its basic units. |
| Approach: | They propose to revisit monosemanticity from the feature decorrelation perspective and advocate for its encouragement. |
| Outcome: | The proposed method improves representation diversity and activation sparsity and improves preference alignment performance. |